ALGORITHM OF AMSGrad AND CHAOS OPTIMIZATION IN MULTILAYERED NEURON NETWORKS WITH STOCHASTIC GRADIENT DESCENT

In this paper, the AMSGrad stochastic optimization method was tested using logistic function that describes doubling process and Fourier spectra of error function. The implementation gradient descent algorithm, AMSGrad, done for a multilayer neural network with hidden layers. program recognizing printed digits written Python software environment. array each digit consisted set "0" "1" size 4x7. sample contained 5 possible distortions 3 arrays did not correspond to any digits. analysis influence value hyperparameters beta1 , beta2 learning rate on optimizing teaching network, which layers 28 neurons each, carried out. We constructed branching diagrams based these parameters. found hyperparameter contribution linear function, is associated number local global minima in retraining network. square contribution, block structure formation blocks processes. If alpha greater than rate, there transition chaotic state, accompanied by both multiple passage through minimum and, apparently, appearance minima. At such speed learning, optimizer practically does work, but presence i.e. gradient, general picture chaos described application shown lead better compared conventional even at optimal (the when existing doubles). Keywords: methods, diagrams.